Discriminative Learning Methods for Interdependent Decision Problems in Natural Language Processing

Many natural language processing (NLP) tasks involve multiple inputs and/or outputs, which are strongly correlated to each other. We call such problems “interdependent decision problems (IDPs)” since the decisions on outputs should be interdependent to each other. In this thesis, we focus on three typical IDPs in NLP and investigate discriminative learning methods for these problems. The first IDP is sentence selection, which is the process of selecting sentences from a document according to some criterion. Sentence selection is an IDP in the sense that whether an item is selected depends on other items in the input set and cannot be decided only from the features of the item. In Chapter 2, we proved that selection problem can be converted into classification problem. Then a new learning algorithm, Selection SVM, is proposed to solve successive selection problems. Experimental results on an artificial dataset and a sentence selection dataset are also reported. The second IDP is multi-topic text categorization, which is a labeling process of assigning all (possibly multiple) relevant topics to a text. Multi-topic text categorization is an IDP decision problem since topics often show strong correlation among them. In Chapter 3, we address the problem of multi-topic text categorization. First We propose a new learning algorithm, Maximal Margin Labeling (MML), and also describe efficient algorithms for MML. MML is tested on datasets of Web pages, and the results are reported. The third IDP is sequence tagging, which is a process of assigning a tag from given tag set to each word in a sequence. Sequence tagging can be called ∗Doctoral Dissertation, Department of Information Processing, Graduate School of Information Science, Nara Institute of Science and Technology, NAIST-IS-DD0461007, August 24, 2006.